8 research outputs found
SQ Lower Bounds for Learning Bounded Covariance GMMs
We study the complexity of learning mixtures of separated Gaussians with
common unknown bounded covariance matrix. Specifically, we focus on learning
Gaussian mixture models (GMMs) on of the form , where and for some . Known learning
algorithms for this family of GMMs have complexity . In
this work, we prove that any Statistical Query (SQ) algorithm for this problem
requires complexity at least . In the special case
where the separation is on the order of , we additionally obtain
fine-grained SQ lower bounds with the correct exponent. Our SQ lower bounds
imply similar lower bounds for low-degree polynomial tests. Conceptually, our
results provide evidence that known algorithms for this problem are nearly best
possible
Information-Computation Tradeoffs for Learning Margin Halfspaces with Random Classification Noise
We study the problem of PAC learning -margin halfspaces with Random
Classification Noise. We establish an information-computation tradeoff
suggesting an inherent gap between the sample complexity of the problem and the
sample complexity of computationally efficient algorithms. Concretely, the
sample complexity of the problem is . We start by giving a simple efficient algorithm with sample
complexity . Our main result is a lower
bound for Statistical Query (SQ) algorithms and low-degree polynomial tests
suggesting that the quadratic dependence on in the sample
complexity is inherent for computationally efficient algorithms. Specifically,
our results imply a lower bound of on the sample complexity of any efficient SQ learner or
low-degree test
Learning general halfspaces with general Massart noise under the Gaussian distribution
We study the problem of PAC learning halfspaces on with
Massart noise under the Gaussian distribution. In the Massart model, an
adversary is allowed to flip the label of each point with unknown
probability , for some parameter . The goal is to find a hypothesis with misclassification error of
, where is the error of the target
halfspace. This problem had been previously studied under two assumptions: (i)
the target halfspace is homogeneous (i.e., the separating hyperplane goes
through the origin), and (ii) the parameter is strictly smaller than
. Prior to this work, no nontrivial bounds were known when either of these
assumptions is removed. We study the general problem and establish the
following:
For , we give a learning algorithm for general halfspaces with
sample and computational complexity
, where is the bias of the target halfspace .
Prior efficient algorithms could only handle the special case of . Interestingly, we establish a qualitatively matching lower bound of
on the complexity of any Statistical Query (SQ)
algorithm.
For , we give a learning algorithm for general halfspaces with
sample and computational complexity .
This result is new even for the subclass of homogeneous halfspaces; prior
algorithms for homogeneous Massart halfspaces provide vacuous guarantees for
. We complement our upper bound with a nearly-matching SQ lower bound
of , which holds even for the special case of
homogeneous halfspaces.Comment: Revised presentatio